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DETAILED ACTION 

This action is responsive to communication filed 5/18/04. 

Claim Rejections - 35 USC § 102 

1 . The following is a quotation of the appropriate paragraphs of 35 U.S. C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 2 1 (2) of such treaty in the English language. 

2. Claim 57 is rejected under 35 U.S.C 102(e) as being anticipated by U.S. Patent No. 
6,504,546 to Cosatto et al. (hereinafter Cosatto). 

Referring to claim 57, Cosatto discloses a method of displaying an integrated display 
audiovisual presentation that provides an audiovisual presentation comprising at least a selected 
portion and having at least on reference point associated therewith, and having a defined 
orientation. See col. 5, lines 25-40. 

Cosatto provides at least two user images, each having a defined user image orientation 
and each user image having a pixel texture and a user object geometry. See col. 6, lines 7-20 and 
the section describing population of a database for samples of facial features, beginning on col. 
10, line 19. 

Cosatto identifies and selects one of the user images as a selected replacement for the 
selected portion responsive to analysis of the defined reference orientation and the defined user 
image orientations. See col. 5, lines 41-64, which describe how bitmap images from recorded 
video (user audiovisual information) are used to replace parts of the face (replacement parts) at 
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an appropriate location on a face. Also see the description of Pose Estimation, beginning on col. 
9, line 20. 

Cosatto replaces the selected portion with the selected one of the user images responsive 
to the analysis, and to an association determined responsive to correlation of at least one selected 
reference point of the reference object with at least one selected replacement point of the one 
user image. See Fig. 7, which shows facial features integrated into a separate base face, 
replacing the previous facial features. Also, see col. 6, lines 7-20. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

4. Claims 1-15, 17-45, 56, and 58 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over U.S. Patent No. 6,504,546 to Cosatto et al. (hereinafter Cosatto) and U.S. Patent No. 
6,307,568 to Rom. 

Referring to claims 1, 26, and 37, Cosatto discloses a system and method for producing 
an integrated display audiovisual presentation with a source of first audiovisual presentation. 
See col. 3, lines 34-53, which describe how video is taken of a model along with an audio track. 
The first audiovisual presentation comprises at least one reference point and at least one 
reference object, wherein at least a selected one reference point is associated with a location on a 
selected one reference object having a placement and orientation. See col 5, lines 25-55, which 
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describe how facial parts are associated with their appropriate location on the face in accordance 
with an orientation. 

Cosatto discloses a source of user audiovisual information, wherein the user audiovisual 
information comprises pixel texture and user object geometry information for user objects 
representative of user images (see col. 6, lines 6-50, which describe using sample bitmaps of a 
user's facial parts for texture mapping within the geometry of the face), the user object geometry 
information comprising at least one replacement point and at least one replacement object image, 
wherein at least a selected one replacement point is associated with a location on a selected 
replacement object image. See col. 5, lines 41-64, which describe how bitmap images from 
recorded video (user audiovisual information) are used replace parts of the face (replacement 
parts) at an appropriate location on a face. 

Cosatto discloses replacing part of the audiovisual presentation with an object from the 
user object audiovisual information based on the selected reference point (i.e. lips) and 
composing means responsive to the first audiovisual presentation, the pixel texture, and the user 
object information for reconstructing an appearance of the user object in the same placement as 
the selected one reference object to provide an integrated display audiovisual presentation, 
wherein at least a selected portion of the user image replaces a respective selected portion of the 
first audiovisual presentation. See Fig. 7, which shows facial features integrated into a separate 
base face, replacing the previous facial features. Also, see col. 6, lines 7-20. 

Although Cosatto discloses replacing a reference object with the best fit sample of user 
object information (i.e. col. 4, lines 15-22) and smoothing and blending the samples which hints 
to transforming the geometry of the user object information for a smooth transition into the first 



Application/Control Number: 09/723 , 1 69 Page 5 

Art Unit: 2173 

audiovisual presentation (col, 3, lines 34-53), Cosatto does not explicitly disclose processing 
means responsive to the first audiovisual presentation and the user audiovisual information, for 
geometrically transforming the user object geometry information to provide transformed user 
object information responsive to the selected one reference point. 

However, Rom discloses a method of replacing reference objects (i.e. images of clothing 
originally on the body) in a presentation with object information (i.e. images of clothing to be 
"tried on"), and is thus similar to Cosatto image replacement and integration method. See Rom at 
col 1, lines 45-63. Rom discloses using critical points similar to the reference points of Cosatto, 
wherein the critical points of Rom are used to adjust the spatial configuration of the clothing 
(object information) to fit the body of each user (reference object). The spatial configuration of 
Rom determines the geometry of the clothing (object information); therefore, adjusting the 
spatial configuration is a geometrical transform of object geometry information to fit the 
geometry and orientation of the reference object (i.e. body), wherein the object information is 
placed on the reference object and integrated (forms a composite) into the original audiovisual 
presentation. See Rom at col. 2, lines 20-33 and col. 4, lines 1-11. 

It would have been obvious to one of ordinary skill in the art with the teachings of 
Cosatto and Rom before him at the time of the invention to modify the image substitution and 
integration method of Cosatto to include geometrically transforming (i.e. adjusting spatial 
configuration of) the user object geometry information responsive to the reference object, as 
taught by Rom, in order to achieve a more accurate, realistic model in the animation, wherein the 
user information smoothly blends into the original presentation (i.e. base model). 
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Referring to claims 9-10, the first audiovisual presentation of Cosatto comprises a time- 
ordered sequence of video images digitized and encoded into an encoded digital signal, that 
comprises a plurality of Motion Picture Expert's Group motion vectors, wherein the processing 
means is further responsive to at least a selected one of the plurality of MPEG motion vectors. 
See col. 53-56, which discusses how the facial parts are handled by a model for the MPEG4 
subsystem. 

Referring to claim 2, the first audiovisual presentation of Cosatto further comprises a 
video input signal See col. 3, line 38 and col 4, line 15. 

Referring to claim 3, the video input signal of Cosatto is a time-ordered sequence of 
video images. See col 6, line 66 - col 7, line 3. Also, the animations of the video signal are 
time-ordered to match the audio track. 

Referring to claims 4, 1 1, 27, and 38, the processing means of Cosatto further comprises 
recognition means responsive to the video input signal, for recognizing at least a selected one 
reference object within at least a selected one video image (the first audiovisual presentation; 
encoded digital signal) of the time-ordered sequence. Col 7, line 14 - col. 8, line 15 describes 
the recognition system of Cosatto, including how facial features (selected reference objects) are 
located within the video. 

Referring to claims 5, 12, and 28, the recognition means of Cosatto further recognizes at 
least one reference point on the associated reference object. See col. 7, lines 33-39. 

Referring to claims 6, 29, and 39, the recognition means of Cosatto comprises image 
recognition means. See col 7, line 14 - col 8, line 15. 
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Referring to claims 7-8, the first audiovisual presentation of Cosatto further comprises 
reference object information that identifies and is associated with a selected one reference object. 
See col. 7, lines 40-61, which describe how reference object information (i.e. color and texture 
analysis, size and position - reference point) is associated with and identifies facial features 
(selected reference objects). 

Referring to claims 13, 30, and 40, Cosatto discloses the processing means further 
comprises first-order object transformation means for transforming at least a selected one 
recognized reference point responsive to the selected one of the plurality of MPEG motion 
vectors. See col. 13, lines 60-65 which discuss morphing (transforming) and col. 14, lines 53-66, 
which discusses how the facial parts are handled by a model for the MPEG4 subsystem. 

Referring to claims 14-15, 31, and 41, the compositing means/reconstructing of Cosatto 
further comprises image transform means (i.e. morphing; col. 13, lines 60-65) for transforming 
the selected one replacement object image (facial feature) associated with the selected one 
reference object. 

Referring to claims 17-18 and 32, the processing means of Cosatto further comprises 
recognition means for identifying a selected one reference object within the first audiovisual 
presentation (col. 7, lines 40-61) and the compositing means further comprises geometric 
transformation means for geometrically transforming the user object geometric information 
responsive to the recognition means. See col, 10, lines 37-53 and col. 13, lines 22-34. 

The compositing means of Cosatto and Rom, supra, further comprises geometric 
transformation means for geometrically transforming the user object geometric information 
producing transformed user object geometric information, responsive to the selected one 
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reference object. See col 6, lines 7-20, where Cosatto acknowledges the ability the change the 
shape and orientation of objects. Also, see Rom at col. col. 1, lines 45-63. 

Referring to claims 19-20, 33, and 43, the replacement object image of Cosatto comprises 
texture map image data representative of at least one selected user object as viewed from at least 
one predefined viewing position relative to the selected user object and the compositing means 
selects a portion of the texture map image data responsive to the processing means. See col 6, 
lines 7-12. Also, see col. 1, lines 62-64. 

Referring to claims 21-22 and 34, the compositing means of Cosatto and Rom, supra, 
further comprises geometric transformation means for geometrically transforming the user object 
geometric information producing transformed user object geometric information, responsive to 
the selected one reference object. See col. 6, lines 7-20, where Cosatto acknowledges the ability 
the change the shape and orientation of objects. Also, see Rom at col. col. 1, lines 45-63. 

The geometric transformation means of Cosatto further comprises mapping means for 
mapping the selected portion of the texture map image data onto the transformed user object 
geometric information producing a replacement object image. See col. 6, lines 7-20. 

Referring to claims 23, 35, and 45, the user audiovisual information of Cosatto further 
comprises a time-ordered sequence of user images and the association means further selectively 
associates a plurality of the time-ordered sequence of user images each associated respectively 
with selected ones of the time-ordered sequence of video images (prior to the replacing). See 
col. 3, lines 44-60, which describe how the user images are selected to coincide with an audio 
track in a time-ordered sequence to provide realistic lip and head movements. 
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Referring to claims 24 and 36, the audiovisual presentation of Cosatto further comprises a 
time-ordered sequence of video masks, and the processing means further comprises selection 
means for selecting a portion of the first audiovisual presentation responsive to at least a selected 
one of the video masks. See col. 15, lines 1-9. 

Referring to claim 25, the selection means of Cosatto is further responsive to at least a 
selected one of the MPEG motion vectors. See col. 14, line 55. 

Referring to claim 42, Cosatto and Rom, supra, discloses the replacement object further 
comprises user object geometric information that is transformed, producing transformed user 
object information, responsive to the recognition of at least one selected reference object within 
the first audiovisual presentation, prior to the replacing. See Cosatto at col. 6, lines 7-20, where 
Cosatto acknowledges the ability the change the shape and orientation of objects. Also, see col. 
6, lines 38-45. Also, see Rom at col 1, lines 45-63. 

Referring to claim 44, the selected portion of the texture map of Cosatto is mapped onto 
transformed user object geometric information, prior to the replacing. See col. 6, lines 7-20. 

Referring to claim 56, Cosatto discloses the audiovisual presentation is comprised of a 
plurality of frames (col. 3, lines 38-39, describe how the audiovisual presentation is video, which 
inherently includes a plurality of frames; see also, col. 7, line 1), at least two of the frames 
having a select portion, the system further comprising analysis means for selectively tracking the 
respective selected portions (i.e. facial feature) of a plurality of frames of the audiovisual 
presentation and identifying and selecting corresponding respective ones of the user images. 
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The image integration means is responsive to the analysis means for selectively 
substituting into the audiovisual presentation the corresponding respective ones of the user image 
for the respective selected portions of the plurality of frames (i.e. substituting facial features 
throughout the animation). See col. 3, line 25 - col. 4, line 26 and col. 12, lines 35 - col. 13, line 
35. 

Referring to claim 58, Cosatto discloses tracking the respective selected portions (facial 
features) comprising at least two frames (col. 3, lines 38-39, describe how the audiovisual 
presentation is video, which inherently includes a plurality of frames; see also, col. 7, line 1), 
identifying and selecting corresponding respective ones of the user images, and providing for 
each frame a replacement user image for integration into the audiovisual presentation responsive 
to the texture map and the user object geometry, and substituting the corresponding respective 
ones of the user image for the respective selected portions of the plurality of frames into the 
audiovisual presentation. See col. 3, line 25 - col. 4, line 26 and col. 12, lines 35 - col. 13, line 
35. 

Although Cosatto discloses replacing a reference object with the best fit sample of user 
object information (i.e. col. 4, lines 15-22) and smoothing and blending the samples which hints 
to transforming the geometry of the user object information for a smooth transition into the first 
audiovisual presentation (col. 3, lines 34-53), Cosatto does not explicitly disclose geometrically 
transforming the user object geometry information to provide transformed user object 
information responsive to the defined orientation. 
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However, Rom discloses a method of replacing reference objects (i.e. images of clothing 
originally on the body) in a presentation with object information (i.e. images of clothing to be 
"tried on"), and is thus similar to Cosatto's image replacement and integration method. See Rom 
at col. 1, lines 45-63. Rom discloses using critical points similar to the reference points of 
Cosatto, wherein the critical points of Rom are used to adjust the spatial configuration of the 
clothing (object information) to fit the body of each user (orientation). The spatial configuration 
of Rom determines the geometry of the clothing (object information); therefore, adjusting the 
spatial configuration is a geometrical transform of object geometry information to fit the 
geometry and orientation of the reference object (i.e. body), wherein the object information is 
placed on the reference object and integrated (forms a composite) into the original audiovisual 
presentation. See Rom at col 2, lines 20-33 and col. 4, lines 1-11. 

It would have been obvious to one of ordinary skill in the art with the teachings of 
Cosatto and Rom before him at the time of the invention to modify the image substitution and 
integration method of Cosatto to include geometrically transforming (i.e. adjusting spatial 
configuration of) the user object geometry information responsive to the defined orientation, as 
taught by Rom, in order to achieve a more accurate, realistic model in the animation, wherein the 
user information smoothly blends into the original presentation (i.e. base model). 

Response to Arguments 
Declarations 

5. The Declarations filed on 5/18/04 under 37 CFR 1.131 has been considered but are 
ineffective to overcome the Cosatto et al. and Rom references. 
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Neither conception nor due diligence have been clearly established based on the evidence 
submitted. 

Conception 

The MPEP states at 715.07 that, 

The affidavit or declaration and exhibits must clearly explain which facts or data 
applicant is relying on to show completion of his or her invention prior to the particular 
date. Vague and general statements in broad terms about what the exhibits describe along 
with a general assertion that the exhibits describe a reduction to practice "amounts 
essentially to mere pleading, unsupported by proof or a showing of facts" and, thus, does 
not satisfy the requirements of 37 CFR 1.131(b). In re Borkowski, 505 F.2d 713, 184 
USPQ 29 (CCPA 1974). Applicant must give a clear explanation of the exhibits pointing 
out exactly what facts are established and relied on by applicant . 505 F.2d at 718-19, 184 
USPQ at 33. See also In re Harry, 333 F.2d 920, 142 USPQ 164 (CCPA 1964) 
(Affidavit "asserts that facts exist but does not tell what they are or when they 
occurred.")- (Emphasis added) 

Applicant provides a conclusory statement (page 1, paragraph 3) in each declaration that 
the claimed subject matter was conceived prior to the filing date of U.S. Patent No. 6,307,568 to 
Rom, 10/28/1998, and prior to the filing date of U.S. Patent No. 6,504,546 to Cosatto et al., 
2/8/2000, then paragraphs 4-13 of each declaration are directed to diligence rather than 
conception. 
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The declaration provides general statements as to what each exhibit is, but makes no clear 
statements as to which parts of the exhibits are relied upon to show claimed features. There is no 
lining up of the facts in the declaration to the claims. 

Applicant provides Exhibits A-H as an appendix. Exhibit A is a listing of computer files 
and their respective dates of modification, Exhibit B is a draft specification last modified 
2/4/1998, Exhibit C is a draft claim and rough outline of a draft specification last modified on 
7/6/1998, Exhibit D is draft claims, Exhibit E is draft claims and a rough outline of a draft 
specification, Exhibit F is a draft specification last modified on 12/24/1998, and Exhibits G and 
H are draft claims. 

U.S. Patent No. 6.307,568 to Rom 

Therefore, Exhibit A is directed to when the computer files were last modified and not to 
the claimed subject matter. Exhibits C, D, and E are insufficient for showing conception because 
draft claims and rough outlines do not show one of ordinary skill in the art how to make and use 
the invention. Exhibits F-H were created after the filing date of Rom, and therefore cannot be 
used in establishing conception before Rom. That leaves Exhibit B as the only evidence 
provided to establish conception prior to Rom. 

The evidence provided (i.e. Exhibit B) appears directed to and may support the claimed 
invention if provided with a properly executed affidavit. However, the current declaration does 
not clearly explain the facts relied upon with respect to the claims. 

U.S. Patent No. 6.504.546 to Cosatto et aL 

Exhibit A is directed to when the computer files were last modified and not to the 
claimed subject matter. Exhibits C, D, E, G, and H are insufficient for showing conception 
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because draft claims and rough outlines do not show one of ordinary skill in the art how to make 
and use the invention. That leaves Exhibits B and F as the only evidence provided to establish 
conception prior to Cosatto. 

The evidence provided (i.e. Exhibits B and F) appears directed to and may support the 
claimed invention if provided with a properly executed affidavit. However, the current 
declaration does not clearly explain the facts relied upon with respect to the claims. 

It is the presently presented claims that define the invention, therefore, it must be clearly 
established that the invention as specifically defined by the present claims was conceived prior to 
the effective dates of the applied references. 

Furthermore, the evidence provided does not appear to support several of the dependent 
claims, for example, claims 10, 13, 24, 25, 30, 36, and 40. 

Diligence 

According to MPEP 715.07(a), 

In determining the sufficiency of a 37 CFR 1.131 affidavit or declaration, diligence need 
not be considered unless conception of the invention prior to the effective date is clearly 
established, since diligence comes into question only after prior conception is established. 
Ex parte Kantor, 177 USPQ 455 (Bd. App. 1958). 

While conception is not clearly established, as described above, a cursory review of 
diligence is provided in the interest of compact prosecution. 
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In an effort to swear behind U.S. Patent No. 6,307,568 to Rom and U.S. Patent No. 
6,504,546 To Cosatto et al., applicant provides Exhibit B, a draft specification for the current 
application, which according to Exhibit A and applicant's statement was last modified on 
February 4, 1998. This is a period of nearly 34 months from the time of first draft to the time of 
filing, November 27, 2000. Applicant states in paragraph 4 that applicant had a continuous 
backlog of cases, but applicant provides no proof that these other cases were prosecuted along a 
similar time frame (i.e. 34 months from first draft to filing). Applicant states that during this 
time, there were 27 drafts of disclosure. However, with the applicant being both the sole 
inventor and the attorney, no correspondence was required in drafting the application, and 
applicant provides no explanation as to why so many drafts were necessary. 

U.S. Patent No. 6.307,568 to Rom 

In establishing priority over U.S. Patent No. 6,307,568 to Rom, the critical period for 
showing diligence is between the filing date of Rom, October 28, 1998 and the filing date of the 
present application, November 27, 2000. Exhibit A is a listing of Applicant's computer files 
regarding the present case. According to Exhibit A, Exhibit E is the last drafted disclosure 
shown before the critical period. In paragraphs 1 1-12 of the declaration, applicant describes 
Exhibit G, modified on December 30, 1998 and Exhibit H, last modified on January 6, 2000. 
Exhibit A shows that no other exhibits are provided between these two dates. This leaves at lest 
one year of inactivity. Furthermore, according to Exhibit A, there is no activity shown between 
January of 2000 and late November of 2000, a period of ten months. This is an insufficient 
showing of due diligence prior to October 28, 1998, and therefore insufficient for priority over 
U.S. Patent No. 6,307,568 to Rom. 
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U.S. Patent No. 6,504,546 to Cosatto et al. 

In establishing priority over U.S. Patent No. 6,504,546 to Cosatto et al, the critical period 
for showing diligence is between the filing date of Cosatto, February 8, 2000 and the filing date 
of the present application, November 27, 2000. According to Exhibit A, Exhibit G, last modified 
on January 6, 2000 is the last drafted disclosure shown before the critical period. According to 
Exhibit A, there is no activity shown between January of 2000 and late November of 2000, a 
period often months, which is insufficient for showing due diligence prior to February 8, 2000, 
and therefore similarly insufficient for priority over U.S. Patent No. 6,504,546 to Cosatto et al. 

Applicant is directed to MPEP 2138.06, which states, 

THE ENTIRE PERIOD DURING WHICH DILIGENCE IS REQUIRED 
MUST BE ACCOUNTED FOR BY EITHER AFFIRMATIVE ACTS OR 
ACCEPTABLE EXCUSES 

An applicant must account for the entire period during which diligence is required. Gould 
v. Schawlow, 363 F.2d 908, 919, 150 USPQ 634, 643 (CCPA 1966) (Merely stating 
that there were no weeks or months that the invention was not worked on is not 
enough.); In re Harry, 333 F.2d 920, 923, 142 USPQ 164, 166 (CCPA 1964) 
(statement that the subject matter "was diligently reduced to practice" is not a showing 
but a mere pleading). 

And, 

The period during which diligence is required must be accounted for by either affirmative 
acts or acceptable excuses. Rebstock v. Flouret, 191 USPQ 342, 345 (Bd. Pat. Inter. 
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1975); Rieser v. Williams, 225 F.2d 419, 423, 1 18 USPQ 96, 100 (CCPA 1958). 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Shawn M. Becker whose telephone number is (703) 305-7756. 
The examiner can normally be reached on M-F 8:30-6:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John W. Cabeca can be reached on (703) 308-3 116. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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